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II  -  SCOPE 

This  Guide  is  the  product  of  an  adopted  and  implemented  suggestion 
(AR  672-20)  and  is  offered  as  an  elective,  nonmandatory  tool  for  application 
to  a  wide  range  of  systems  acquisition/evaluation  tasks,  by  any  U.S.  Government 
element. 
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III  -  OVERVIEW 

A.  EASIS  OF  NEED. 

Despite  the  many  CA/EA/RA/COEA/CTEA  governing  reg.  lations  and  resalting 
multitude  of  sophisticated  decision-analytic  computer  models,  a  gap  continues 
to  exist  in  terms  of  tools  to  meet  everyday  needs.  For  instance: 

1.  When  there  is  not  enough  data  available  to  fully  exercise  a  "canned" 
model . 

2.  When  there  is  not  enough  time  and/or  lack  of  available  operations 
research  or  computer  assistance  personnel  to  meet  suspense  dates. 

3.  When  there  is  need  to  deal  with  a  combination  of  tangible  (more  or 
less  quantifiable)  and  intangible  (primarily  subjective)  considerations  in 
evaluating  a  given  option  or  alternative. 

4.  When  there  is  need  for  systematic  evaluation  of  a  single  course  of 
action,  by  comparison  with  a  known,  accepted  standard  or  product.  (Most 
existing  models  accommodate  comparison  of  multiple  alternatives,  but  do  not 
provide  for  evaluating  the  relative  merits  of  a  single  option). 

5.  When  there  is  need  to  compare  very  dissimilar  items  in  terms  of 
relative  worth  for  similar  roles. 

6.  When  there  is  need  for  preliminary  screening  of  alternatives,  either  — 

a.  To  reduce  a  large  number  of  options  to  a  more  manageable 
number/range  on  which  full-blown,  more  formal  analysis  can  be  conducted;  or  — 

b.  To  identify  which  facets  are  most  critical  to  the  evaluation  and 
which  therefore  warrant  further,  in-depth  analysis. 


PRECEDING  PAGE  ElaNK-MOT  FILMED 

1  _  . 

B.  CONCEPT. 

The  core  of  the  CBRA  technique  is  drawn  from  ordinary  principles  of 
decision  analysis/source  selection  evolution,  in  terras  of  problem  definition 
and  identification  and  exploration  of  alternatives. 

1.  The  chief  advantage  of  CBRA  is  that  it  assimilates  these  "universal" 
evaluation  methods  into  a  simple  but  highly  adaptable  tool  to  imprcve  the 
depth,  quality,  and  documentation  of  analyses  that  would  otherwise  rely  on 
totally  subjective  processes  —  or  not  be  attempted  at  all. 

a.  Permits  assessment  of  a  wide  range  of  factors  in  evaluation  processes 
by  correlating  both  tangible  and  intangible  aspects  and  expressing  them  in 
quantifiable  terms  conducive  to  both  objective  and  subjective  evaluation. 

b.  Designed  for  individual  or  group  use,  large  or  small  number  of 
variables  —  no  special  training  required  for  either  manual  or  automated 
appl ication . 

c.  With  computer  assistance,  permits  accommodation  of  a  large  volume  of 
data  in  a  practical  visible  manner;  featuring  — 

(1)  Self-contained,  English-language  program  4n  ready-to-use  status 
on  any  terminal  with  S&E  computer  access;  no  prior  training  or  special  skills 
requ ired . 

(2)  Responsive,  see-as-you-go  results.  Can  be  used  either  in 
"building  block"  increments,  or  completed  at  a  single  sitting. 

(3)  Allows  real-time  participation  from  multiple  locations  via  remote 
access  terminals;  and  in  most  cases,  permits  hard  copy  of  input/output  and 
other  documentation  if  desired. 

(4)  Source  program  (Appendix  D)  is  easily  adaptable  to  non-IBM/S&E 
computers  if  desired  (see  IV. E. 2). 


2.  Once  the  problem  (task)  has  been  defined,  the  following  generally 
applies : 

a.  Identify  principal  characteristics  which  define  the  benefits  and 
drawbacks  of  the  candidate  item(s)  or  alternative(s)  against  the  overall 
objective . 

b.  Obtain  assessment  from  key  participants  as  to: 

(1)  Relationship  of  the  critical  parameters  to  each  other  (within 
the  framework  of  the  overall  task) . 

(2)  Weight  or  ranking  of  the  critical  parameters  (against  a  total 
weight  of  100) . 

(3)  Estimated  worth  or  rating  of  each  candidate,  for  each 
critical  characteristic  above,  expressed  as  a  numeric  rating  on  a  decimal 
scale  such  as  1-10. 

c.  Compute  candidate  ratings  against  critical  characteristic  weights 
to  derive  candidate  weighted  score,  which  yields  a  projected  "unit  worth"  in 
quantifiable  terras. 

d.  Obtain  cost  estimates  for  each  candidate  and  express  in  common 

terras. 

e.  Divide  each  candidate  cost  by  the  corresponding  candidate  weighted 
score,  to  yield  a  numeric  cost  per  "benefit",  or  Cost-Benefit  Ratio  (CBR) . 

f.  Expanded  discussion  of  the  technique,  tailored  to  a  variety  of 
uses,  plus  case  studies/exaraples,  are  presented  in  Section  IV  following. 


' 

t 

3.  A  comment  about  other  principles,  techniques,  terms,  or  tools  within  the 
broad  category  of  decision  theory/analysis:  While  such  topics  as 
Mul ti-Attr ibu te  Utility  Theory,  Utility  Curves,  use  of  Ratio  vs.  Absolute 
Scale  —  endless  etc.  —  lie.  outside  the  purview  of  this  Guide,  there  are  no 
known  incorapatiblities  or  barriers  to  using  CBRA  in  conjunction  with  such 
tools.  Quite  simply,  CBRA  is  designed  to  assist  in  task  layout,  scorekeeping 
and  feedback  during  the  analysis;  and  if  Utlity  Curves  (etc.)  are  desired, 
they  simply  become  the  basis  of  the  scoring  standard  ("yardstick")  selected 
for  use  in  the  evaluation  process. 


-\r>  ■ 

i- 
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C.  SUGGESTED  USES. 

1.  To  illustrate  potential  use  of  CBRA  against  the  kinds  of  needs 


previously  discussed,  the  following  sampl 

a.  Assess  projected  benefit 
vs  cost  of  a  proposed  ECP  or 
PIP;  only  skeletal  in  form  t  ion 
is  available  at  this  point. 

(Ref.  III.A.l) 

b.  "What  If"  question;  one- 
day  reaction  time:  Wliat  is 
the  estimated  impact  in  terms 
of  "bang  for  bucks"  if  you 
augment  the  currently  planned 
payload  capability  of  System  X 
to  accommodate  expanded  mission 
equipment  package? 

(Ref.  III. A. 2) 

c.  Compare  alternative  mission 
equipment  packages,  in  terms  of 
projected  performance  capability 
vs.  human  factors  considerations. 
(Ref.  III. A. 3) 


tasks  are  outlined: 

Remarks 

Analysis  results  will  help 
determine  whether  formal  review 
and  staffing  will  be  pursued. 

Outcome  will  Influence  decision 
on  whether  to  submit  request  for 
a  program  increment,  FY  XX-XX 
POM  cyt  le . 


Results  will  influence  scenario 
outline  and  task  selection  for 
Mission  Profile  Operational 
Test/Evaluation  Plan  (part  of 
COEA) . 


17 


d.  Determine  win- tiler  It  Is  cost 
effective  (additional  hang  for 
additional  bucks)  to  modernize 
older,  fielded  assets  to  <k>w 
system  configuration,  to 
achieve  "homogeneous"  fleet. 

(Ref.  III. A. 4) 

e.  Explore  relative  merits  of 
ground  vs.  aerial  vehicles  for 
laser  designation  of  targets, 
in  various  support  roles;  e.g., 
field  artillery,  air  cavalry, 
attack  helicopter,  command  and 
control . 

(Ref.  III. A. 5) 

f.  Screen  a  do2en-plus  initial 
options  and  narrow  down  to  those 
which  reflect  greatest  potential 
for  and/or  greatest  differences 
in  type  of  approach  to  task 
solution;  isolate  critical  aspects 
on  which  further  analysis  should 
be  focused . 

(Ref.  III. A. 6) 


Outcome,  If  positive ,  will  be 
incorporated  in  justification  of 
agency  proposal  for  a  conversion 
pr  og  r  am . 


Results  will  be  fed  into  Trade 
Off  Analyses/Determinations 
(TOA/TOD)  during  Concept 
Formulation  Package  (CFP) 
prepara  tion . 


Results  will  help  frame  the 
structure  and  scope  of  subsequc..; 
COEA. 


Specific  instructions  and  examples  of  CBRA  application  ore  included  in 


IV  -  OPERATING  INSTRUCTIONS 


THE  PROCESS 


Basic  Steps* 


Discussion 


l.  Define  the  task.  No  mi  t ter 


what  the  nature  of  trie  exercise. 


the  first  step  is  to  establish  the 
objective  (define  the  problem). 


2.  Identify  Critical  Parameters. 


The  next  step  is  to  determine  what 
key  elements  or  principal  charac¬ 
teristics  will  "make  or  break"  a 


This  forms  the  summary  level 
(top  piece)  of  the  evaluation 


framework . 


These  critical  parameters  (CP) 


form  the  backbone  of  the 
evaluation  structure,  and 
every  CP  occupies  a  "cell" 


solution  to  the  problem;  evaluation  within  the  overall  structure, 

"musts".  When  selecting  critical 
parameters,  try  to  assure: 

a.  Focus  on  aspects  which  are  important  in  terms  of  contribution  (pro 
and  con)  to  the  final  purpose/objective  of  the  task;  i.e.,  the  things  which 
should  influence  a  decision. 

b.  Completeness  of  coverage — so  that  the  primary  benefits  and 
drawbacks  of  the  potential  candidate  itera(s)  or  approach(es)  are  Included  in 
the  scope  of  evaluation. 

c.  Recognition  of  interdependence  among  factors  (such  as  target 
recognition  as  a  function  of  target  de tec tion) ;  but  at  the  same  time, 
avoidance  of  direct  redundancy  (e.g.,  evaluation  of  airspeed  both  as  a 
technical  characteristic  and  as  an  indicator  of  maneuverability).  To  avoid 
this  kind  of  redundancy,  tailor  the  parameters  to  the  function  or  capability 
desired,  rather  than  the  means  of  attaining  it. 


*Be  sure  to  document  assumptions,  sources,  and  rationale  thrcxighout  the 


process. 


d.  When  cixnparing  alternative  candidates,  assure  tliat: 

(1)  Emphasis  is  given  to  aspects  in  which  the  candidates  differ; 
inclusion  of  too  much  similarity  tends  to  "muddy"  the  outcome  rather  than 
enhance  discrimination. 

(2)  There  is  basis  for  meaningful  comparison  between/among 
candidates  against  each  parameter  selected.  However,  if  the  candidates  are  so 
dissimilar  that  their  characteristics  or  features  cannot  be  compared,  then 
reorient  the  evaluation  to  assess  their  respective  contribution  to  the  end 
purpose,  rather  than  "hardware”  aspects. 

e.  If  a  single  option  is  to  be  evaluated  against  an  established 
"yardstick",  it  can  be  done  either  in  terras  of  comparable  aspects,  or  in  terras 
of  added/delta  capability  or  benefits  for  added/delta  cost. 

f.  Some  typical  critical  parameters: 

(1)  Mission  capability — including  speed,  endurance,  firepower. 


etc. 


(2)  Survivability — encompassing  delectability  ,  crashworthiness, 
ballistic  protection,  EW  countermeasures,  etc. 

(3)  Operational  suitability — addressing  human  factors,  handling 
qualities,  mission  analysis,  etc. 

(4)  Logistics/Management — including  RAM,  availability, 


provisioning,  "-ilities",  etc. 

3.  Lay  out  the  framework. 
Variously  referred  to  as  a  blue¬ 
print,  pyramid,  wiring  diagram, 
decision  tree,  hierarchy,  etc. — 
regardless  of  "label",  the 
object  is  to  establish  the  re¬ 
lationship  of  the  critical 
parameters  to  each  other  wLthln 
the  overal 1  task. 


This  rough  diagram  depicting 
critical  parameter  relationships 
becomes  the  evaluation  structure 
(ES) .  May  range  from  very 
simple  to  very  complex,  dependm0 
on  the  nature  of  the  task  itseli. 
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4.  Allocate  weights  reflecting 
the  relative  Importance  or 
"preference"  of  the  critical 
parameters;  total  weLght  for  the 
cells  within  a  given  block  should 
equal  100X. 

Illustration: 


Referred  to  as  cell  weight  (CW) 
In  a  multi-level  evaluation 
structure,  each  group  of 
subordinate  cells  with  their 
immediately  superior  cell  form 
a  cell  block  (CB).  Every  cell 
receives  a  weight. 


''  I 

l!  i 
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These  scoring  standards  (SS) 
become  a  sort  of  "yardstick"  to 
facilitate  consistent,  objective 
assessment. 


a.  Obviously,  wherever  possible,  it  is  desirable  to  use  specific 
requirements  (from  ROC,  system  specification,  test  plans,  etc.)  to  establish 
benchmarks;  including  "required"  and  "desired"  characteristics. 

b.  In  the  absence  of  specific  tangible  requirements  as  a  basis  for 
measurement,  construct  narrative  criteria  which  describe  as  definitively  as 
possible  what  will  be  considered  good/acceptable,  bad/unacceptable,  and 
better/desirable  in  evaluating  a  given  parameter. 

c.  In  some  instances,  a  different  kind  of  yardstick  may  be  needed; 
e.g.,  where  detail  criteria  is  not  feasible  but  where  a  known,  accepted 
product  can  be  used  as  a  standard,  with  proposed  options  compared  against  it. 

d.  Illustrations  of  various  yardstick  are  included  in  the  case 
studies/examples  in  IV. B. 

6.  Verify  candidates  or  alter-  Frequently  implicit  in  task 

natives  to  be  evaluated.  definition;  however,  review 

should  at  least  be  made  to  assure 
that  all  reasonable  options  are 
considered . 


S .  Formulate  evaluation  criteria. 
Evaluation  is  essentially  a 
mea sur emen t  process,  using  a 
yardstick  tailored  to  the  specific 
needs  of  the  task.  Even  though 
critical  parameters  predispose 
wha t  will  be  measured,  it  remains 
to  be  determined  how  to  measure 
them  systematically. 
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7  .  Assign  and  Compute  Score: s  . 


a.  Assess  the  estimate*!  worth 
(or  preference  or  ranking)  of  each 
candidate,  for  each  critical 
parameter,  against  Its  scoring 
standard  and  express  as  a  numeric 
value  on  a  decimal  scale  such 

as  1-10  or  1-100. 

b .  Compu  te  cand Ida  te  we igh  ted 
scores,  to  yield  an  estimated 
"candidate  worth"  or  "benefit 
rating"  in  numeric  terras. 

8.  Obtain  cost  estimates  for  each 
candidate  and  express  in  common 
terras. 


9 .  Compute  cost-benefit  ratio. 
May  be  corapu  ted  /c  ompared  for 
individual  cell  blocks  or  for 
intermediate  cell  groupings 
("modules"),  as  well  as  for 
task  summary  level. 


Raw  scores  (RS)  are  applied  only 
to  the  lowest  level  cells  within 
each  block  (those  which  have  no 
subordinate  cells). 


Raw  score  multiplied  by  cell 
weight  yields  the  weighted  score 
(WS) ,  repeated  for  each  ascending 
level  (RS  x  CW  •*  WS).  This  process 
is  referred  to  as  rollup. 

Candidate  cost  (CC)  may  be  drawn 
from  any  type  of  funds  or  any 
category  of  cost,  as  needed/ 
available  to  compare  the 
candidates  in  the  areas  being 
evalua  ted . 

Divide  each  candidate  cost  by  the 
corresponding  weighted  score  to 
yield  cost-benefit  ratio  (CBR) , 
which  is  an  expression  of  re¬ 
lative  "bang  for  bucks". 

(CC  7  WS  -  CBR)  . 
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10.  Analyse  results,  performing 
sensitivity  analyses  wherever 
indicated  by  those  results,  to 
help  solidify  conclusion  and 
support  recommendations  for 
fu  r  ther  ac  t  ion  . 


The  cost-benefit  ratio  with 
supporting  analysis  (CBRA)  assists 
in  decision  making  by  isolating 
issues  and  clarifying  their 
impacts  under  varying  conditions; 
but  does  not  predispose  a  one- 
only  best  answer  or  final 
solution.  It  is  a  decision 
tool — not  a  decision  maker. 


NOTES: 

a.  Throughout  this  Guide,  the  assumption  is  made  that  cost  is  an 
important  consideration  in  most  any  evaluation  or  Justification  task,  and  for 
this  reason  cost  considerations  are  inherent  in  the  Instruction  and  discussion 
material.  However,  if  for  any  reason  it  is  not  pertinent  or  possible  to 
address  cost  issues,  the  technique  may  still  be  used  by  simply  omitting  the 
cost-related  steps  in  the  process. 

b.  Practical  examples  of  the  technique,  tailored  to  different  types  of 


tasks,  are  Included  in  IV.  B. 


B.  WALK-THROUGH  INSTRUCTIONS/ CASE  STUDIES 


1 .  Manual  Usage. 

For  obvious  reasons,  it  is  impractical  to  utilize  CBRA  manually  on 
very  large  or  complex  tasks;  therefore,  instructions  for  manual  applications 
are  geared  to  fairly  simple,  straight  forward  examples, 
a .  System  X  "Shopping  List". 

(1)  Ta  sk:  Screen  the  list  of  potential  items  against  System  X 
modernization  objectives  and  constraints,  to  eliminate  any  obvious  drop-outs 
and  highlight  marginal  items/issuus  which  warrant  further  analysis.  Thrust  of 
modernization  is  on  improved  capability  without  extensive  new  development, 
with  early  fielding  and  at  moderate  cost. 


Candidate  Improvements 

Crew  station 
(modify) 

De-icing  system 
(add) 

Auxiliary  generator 
(add) 

Uprated  transmission 
(modify) 

Teflon  bearings 
(replace) 


Benefits /Drawbacks 

Human  factors,  growth  potential; 

reduced  commonality . 

Adverse  weather  operations;  weight 
and  power  penalty. 

Safety  backup,  power  source  for 
test /maintenance  equipment;  sp...  e 
and  weight  penalty. 

Mission  performance  benefits;  M'.V: 
fuel  consumption  Impacts. 

Improved  RAM;  loss  of  commonality. 


Integrated  communications 

package  (ICP) 

(replace) 

Doppler 

(add) 

Automatic  direction 

£  inder 

(replace) 

Night  vision  system 
(add) 

Infrared  suppressor 
(replace) 

Infrared  Jammer 
(add) 

Laser  warning  device 
(add) 

2.  Critical  Parameters. 

Based  on  the  task  objective  and 
following  are  selected: 

Mission  cpability/perforraance 
Weight/power  requirements 
Human  factors 
Survivability 
Logistic  support 


Saves  weight,  space  &  reduces  crew 
workload;  not  compatible  with 
older,  fielded  communica tlons 
equipment. 

Navigation  aid;  requires  ICP 
above  to  accommodate. 

More  accurate;  loss  of 
commonality. 

Round-the-clock  operations; 
weight  and  space  penalty. 
Survivability;  weight  and  power 
penalty . 

Survivability;  weight  and  power 
penal ty . 

Survivability;  space/ loca tion 
and  electro-mange  tic  problems. 

the  candidate  characteristics, 


imprcvemen  t 


Schedule 


3.  Evaluation  Structure 


To  keep  it  simple  (bearing  in  mind  this  is  a  screening  exercise,  not  a 
"final  selection"),  a  vertical  structure  is  used: 

System  X 

Performance  capability 
Weight/ power  requirements 
Human  factors 
Surv  ivab  ility 
Logistic  support 
Schedule 
4 .  Cell  Weight 

Widely  different  viewpoints  may  exist  concerning  the  importance  of 
these  parameters;  therefore,  even  though  only  one  version  is  employed  in  this 
example,  it  is  advisable  to  run  2  or  3  variations  in  actual  practice,  to 
reflect  anticipated  extremes. 

System  X  (100) 

Perf  Cap  35 

Wt/Pwr  15 


Human  Factors  10 


Surv 


15 


Log  Spt  10 


Schedule 


15 


5 .  Scoring  Standards.  — ' 


On  a  scale  of  1-10,  following  yardstick  Is  established: 


1-4 

5-7 

8-10 

Perf  Cap  (benefit) 

Minor 

Significant 

Highly  sig./ 

Multiple  — 

Wt/Power  (impact) 

Major 

Moderate 

None/  improves 

Human  Factors 

Minor 

Moderate 

Substantial 

(benefit) 

Survivability 

Minor 

Significant 

3/ 

Major/new  — 

Logistic  Spt 

Significant 

Moderate 

None /improved 

Schedule  (target) 

Pacing  item 

Modera  te 

Low  risk  & 

&  h  igh  r  isk 

r  is’.. 

nonpacing  item 

NOTES : 

—  If  warranted,  scoring  standards  can  be 

broken  down  more 

specif ical ly ; 

i.e.,  translating  "major" 

&  "moderate" 

weight  penalties 

into  numbers  o: 

pounds,  schedule  variances 

into  numbers 

of  mon  ths  ,  e  tc  . 

2/ 

—  "Multiple"  benefit  example  »  ICP,  which  permits  accommodation  of  navigation 

items  in  addition  to  its  own  (direct)  benefits. 

3/ 

—  "New"  means  protection  not  previously  provided  even  in  lesser  forms. 

6 .  Verify  Candidates. 

In  this  example,  candidates  are  already  Identified  via  the  "shopping 
list".  However,  in  other  cases,  candidates  are  not  predetermined  and 
need  to  be  solidified  here  after  critical  parameter  identification  and  tri... 
weight  allocation,  but  before  actual  scoring — to  maximize  thoroughness  ana 
objectivity  of  evaluation. 
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7 .  Scoring: 


(a)  Raw  scores.  Using 

be  s  t  da  ta 

available,  a 

ssign  raw  scores 

for  each  candidate 

against  each 

critical  parameter,  in 

accordance  with 

the 

pertinent  scoring 

standard . 

CW 

(.35) 

(.  15) 

(.10) 

(.15) 

(.10) 

(.10) 

Perf 

Wt/ 

Human 

Sa  fe  ty/ 

Log 

Cand  ida  te 

Cap 

Power 

Fac  tors 

Surv- 

Spt 

Schedule 

(RS) 

(RS) 

(RS) 

(RS) 

(RS) 

(RS) 

Crew  sta. 

1 

8 

9 

4 

5 

6 

De-ice 

6 

6 

1 

1 

5 

7 

Aug.  gen 

9 

3 

2 

7 

4 

8 

Uprated  Xmn 

9 

8 

5 

3 

4 

6 

Teflon  brgs 

3 

6 

5 

5 

8 

2 

ICP 

8 

8 

9 

4 

3 

5 

ADG 

6 

4 

7 

2 

5 

6 

MVS 

7 

2 

3 

7 

5 

3 

IR  supp. 

4 

4 

5 

8 

5 

8 

IR  jam. 

5 

3 

5 

9 

3 

4 

Laser  warn. 

5 

5 

5 

7 

4 

6 

Doppler 

7 

4 

6 

7 

6 

7 

i 

’( 

\ 
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Computed  below:  (CW  x  RS  »  WS) 


CW 

Cand  Ida  te 

(. 

Per  f 

Cap 

35) 

(• 

Wt/ 

Power 

15) 

(.10) 

Human 

Factors 

(.15) 

Sa  fe  ty  / 

Surv- 

(.10) 

Log 

Spt 

(.10) 

Schedule 

Total 

(RS) 

(WS) 

(RS) 

(WS) 

(RS) 

(WS) 

(RS) 

(WS) 

(RS) 

(WS) 

(RS) (WS) 

(WS) 

Crew  sta 

1 

.35 

8 

1.20 

9 

.90 

4 

.60 

5 

.50 

6 

.90 

4.45 

De-ice 

6 

2.10 

6 

.90 

1 

.10 

1 

.15 

5 

.50 

7 

1.05 

4.80 

Aug.  gen 

9 

3.15 

3 

.45 

2 

.20 

7 

1.05 

4 

.40 

8 

1.20 

6.45 

Uprated  Xmn 

9 

3.15 

8 

1.20 

5 

.50 

3 

.45 

4 

.40 

6 

.90 

Teflon  brgs 

3 

1.05 

6 

.90 

5 

.50 

5 

.75 

8 

.80 

2 

.30 

I  CP 

8 

2.80 

8 

1.20 

9 

.90 

4 

.60 

3 

.30 

5 

.75 

6.55 

ADG 

6 

2.10 

4 

.60 

7 

.70 

2 

.30 

5 

.50 

6 

.90 

5.10 

NVS 

7 

2.45 

2 

.30 

3 

.30 

7 

1.05 

5 

.50 

3 

.45 

5.05 

IR  supp. 

4 

1.40 

4 

.60 

5 

.50 

8 

1.20 

5 

.50 

8 

1.20 

5.40 

IR  jam. 

5 

1.75 

3 

.45 

5 

.50 

9 

1.35 

3 

.30 

4 

.60 

4.95 

Laser  warn. 

5 

1.75 

5 

.75 

5 

.50 

7 

1.05 

4 

.40 

6 

.90 

4.95 

Doppler  7  2.45  4  .60  6  .60  7  1.05  6  .60  7  1.05  6.3 


8.  Candidate  Cost. 


For  this  example,  estimated  unit  acquisition  cost  is  used  (development 
plus  procurement  cost,  divided  by  the  planned  number  of  systems  to  be 
modernized),  expressed  in  constant  year  dollars.  If  available,  other  costs 
(such  as  O&S)  could  be  included  . 


Cand  ida  te 


Cost  ($-K) 


Crew  sta 


De-ice 


Aux .  gen 
Uprated  Xmn 
Teflon  brgs 


Doppler 


IR  supp. 


IR  jam. 


Laser  warn 


3.7 


9.  Cost-Benefit  Ratio 


Computed 

below:  (CC 

-  ws  = 

CBR) 

Cand  ida  te 

CC  ($-K)  -r 

WS 

*  CBR 

Crew  sta. 

5.6 

A.A5 

1.26 

De-ice 

3.5 

A. 80 

.73 

Aug.  gen 

2.9 

6.A5 

•A5 

Uprated  Xmn 

13. A 

6.60 

2.03 

Teflon  brgs 

5.8 

A. 30 

1 .35 

ICF 

8. 1 

6.55 

1 .2A 

Doppler 

7. A 

6.35 

1.17 

ADF 

5. A 

5.10 

1.06 

NVS 

15.2 

5.05 

3.01 

IR  supp. 

5.8 

5. A0 

1.07 

IR  jam. 

7.6 

A. 95 

l  .5A 

Laser  warn. 

3.7 

A. 95 

.75 

Average  CBR  *« 


(Use  as  SS) 


Review  of  this  first-cut  CBR  "output"  leads  to  several  areas  of 
question/need  for  further  analysis. 

(a)  Since  no  specific  standard  CBR  is  available,  and  the  candidates 
are  widely  spread,  the  average  CBR  of  1.3  can  be  used  as  a  standard.  Thus, 
candidates  with  CBR  of  approx.  1.3  or  less  appear  to  offer  promising  "bang  for 
bucks",  while  those  with  CBR  substantially  greater  than  1.3  deserve  further 
scrutiny.  See  comparative  CBR  annotations  below: 


Candidate 

CC  ($=K)  -r 

ws  « 

CBR  -r 

SS  « 

Comparative  CBR 

Crew  sta. 

5.6 

4.45 

1.26 

(1.31) 

.96 

F 

De-ice 

3.5 

4.80 

.73 

It 

.56 

F 

Aug.  gen 

2.9 

6.45 

.45 

II 

.34 

F 

Uprated  Xmn 

13. A 

6.60 

2.03 

It 

1.55 

U 

Teflon  brgs 

5.8 

4.30 

1.35 

II 

1.03 

M 

ICP 

8.1 

6.55 

1.24 

II 

.95 

F 

Doppler 

7. A 

6.35 

1.17 

II 

.89 

F 

ADF 

5.4 

5.10 

1.06 

II 

.81 

F 

NVS 

15.2 

5.05 

3.01 

II 

2.30 

U 

IR  supp. 

5.8 

5.40 

1.07 

II 

.82 

F 

IR  jam. 

7.6 

4.95 

1.54 

II 

1.18 

M 

Laser  warn. 

3.7 

4.95 

.75 

II 

.57 

F 

F  »*  favorable  comparison  to  standard 
U  *  unfavorable  comparison  to  standard 
M  *  marginal 


(b)  Suggested  directions  to  pursue: 


*  Have  the  right  critical  parameters  been  selected  for  the  task? 

Are  there  omissions/redundancies? 

*  Do  the  individual  scoring  standards  reflect  proper  means  of 
measurement  for  their  respective  critical  parameters? 

*  Are  there  imbalances  in  the  cell  weights?  For  example,  do  one  or 
two  parameters  drive  the  whole  outcome — and  if  so,  should  they? 

*  Do  the  assigned  scores  properly  reflect  the  candidate  benefits/ 
drawbacks?  (This  is  where  clear  documentation  of  rationale  pays  off.) 

*  Are  there  interdependencies  that  were  not  reflected  in  the 
weighting  or  scoring  process?  A  good  cross-check  is  to  group  the  candidate 
items  into  packages — e.g.,  evaluate  items  which  impact  weight/power  with  those 
which  provide  it;  those  which  save  space  with  those  which  require  it — and  see 
if  a  favorable  CBR  results  from  the  combinations  of  related  items. 

*  Are  there  aspects  which  would  logically  be  added  to  the 
evaluation  if  information  were  available?  If  so,  do  a  quick  test  with 
"assumed  data"  to  see  how  it  affects  outcome.  This  may  point  the  way  to 
essential  fact  finding  before  proceeding  further. 
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b.  "Go-No  CiO "  t'or  Hoimiiiciiouns  Fleet  Mtxlorn  iza  tion  Request. 

(1)  Task:  Determine  whether  it  is  cost-beneficial  to  update 
fielded  System  X  assets  (in  varying  configurations)  to  the  fully  raoderized 
System  X  configuration.  Outcome  (if  positive)  will  be  incorporated  in 
justification  of  proposed  conversion  program  submission. 

(2)  Critical  Parameters.  In  this  instance,  the  separate 
configurations  themselves  become  the  critical  parameters,  because  the  central 
question  is  one  of  delta  benefit  vs.  delta  cost  to  update  to  the  fully 
moderized  standard. 

Quantity*  CP 

(Was  1000;  all  have  System  X  =  original  configuration 

been  conver  ted  to 
XI,  X2 ,  X3  or  X4) 


300 

XI 

«  X  plus  weapon  subsystem 

200 

X2 

*  XI  plus  performance  package 

100 

X3 

=  X2  plus  survivability  mode 

400 

X4 

*  X3  plus  fire  control  &  comm/ 

nav  package 

*For  info  only. 
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(3)  Evaluation  Structure.  Layered  structure  is  not  pertinent,  since 


each  critical  parameter  is  to  be  compared  independently  against  the  fully 
moderized  standard . 

(4)  Cell  Weights.  Not  applicable — sane  reason  as  (3)  above. 

(5)  Scoring  Standards.  Only  one  yardstick  is  needed;  1-10  scale  is 
established  by  using  the  original  (X)  configuration  as  the  "l"  mark  and  the 
fully  modernized  (X4)  as  the  "10"  mark.  XI,  X2  and  X3  will  be  measured  by 
comparison  against  this  standard. 

(6)  Verify  Candidates.  In  this  case,  candidates  are  predetermined 
(synonymous  with  critical  parameters)  . 

(7)  Scoring. 

(a)  Raw  score  is  based  on  assessment  of  each  candidate's  "worth" 
in  comparison  to  the  standard. 

Candidate  RS 


X 

XI 
X2 
X3 
X4 


l  (given) 
3 
6 
7 

10  (given) 


(b)  Since  cell  weights  are  not  pertinent,  weighted  score  is 
merely  the  raw  score  translated  into  added  (delta)  benefit  to  be  derived  if 
modernized  to  X4  standards.  Also,  the  Incremental  steps  to  get  from  original  X 
to  XI,  X2,  X3  and  X4  are  shown  (purely  for  reference). 


Candida  te 

RS 

WS 

X 

1 

+9 

(X  to  XI) 

(+2) 

XI 

3 

+  7 

(XI  to  X2) 

(+3) 

X2 

6 

+4 

(X2  to  X3) 

(+D 

X3 

7 

+3 

(X3  to  X4) 

(+3) 

X4 

10 

+0 

(8)  Candidate  Cost.  Since  development  cost  is  essentially  "sunk" 
(accomplished  prior  to  X4  configuration  conversion),  and  quantities  per 
candidate  vary,  unit  procurement  cost  of  modernization  (FY  80  constant 
dollars)  is  selected  as  best  available  basis  for  comparison. 

Cand  id  a  te  Cost  to  Modernize  ($=K) 


X 

1100 

(X  to  XI) 

(550) 

XI 

500 

(XI  to  X2) 

(200) 

X2 

450 

(X2  to  X3) 

(150) 

X3 

350 

(X3  to  X4) 

(300) 

X4 

0 

I 


Cost-Benefit  Ratio. 


Cand  id  a  te 

CC 

MS 

CBR 

X 

1100 

9 

122* 

(X  to  XI) 

(500) 

(2) 

(250) 

XI 

650 

7 

93 

(XI  to  X2) 

(250) 

(3) 

(83) 

X2 

450 

4 

113 

(X2  to  X3) 

(150) 

(1) 

(250) 

X3 

350 

3 

117 

(X3  to  X4) 

(350) 

(3) 

(117) 

X4 

(Same  as 

X,  because 

X4  reflects 

conversion  of  original  to  fully 
modernized  configuration). 


♦Standard 


(10)  Analysis: 


(a)  Comparison  of  candidate  CBR  to  the  "standard"  CBR  reveals  the 
following: 


Candida  te 

CBR 

Std 

Comparative  CBR 

X/X4 

122 

122 

1.00 

"Yardstick" 

(X  to  XI) 

(250) 

122 

(2.05) 

(Unfavorable) 

XI 

93 

122 

.76 

Favorable 

(XI  to  X2) 

(  83) 

122 

(  -88) 

(Fav  orable ) 

X2 

113 

122 

.93 

Favorable 

(X2  to  X3) 

(150) 

122 

(1.23) 

(Marginal) 

X3 

117 

122 

.96 

Fav  orable 

(X3  to  X4) 

(117) 

122 

(  .96) 

(Favorable) 

(b)  Review  indicates  that  some  of  the  incremental  configuration  steps 
(shown  for  reference)  were  less  "cost-beneficial"  than  the  standard,  but  that 
all  current  candidates  for  modernization  do  compare  favorably  to  the  standard. 
While  favorable /unfavorable  comparisons  do  not  necessarily  equate  to  "good 
buy/bad  buy",  they  do  provide  a  basis  for  comnwnica ting  the  relative  "go'od 
buy"  to  decision  makers — by  using  the  known  and  accepted  X4  configuration  as  a 
baseline.  Thus,  having  substantiated  a  rough  cost-benefit  relationship, 
additional  factors  (such  as  logistic  support  advantage  of  a  homogeneous  fleet) 
may  be  introduced  to  amplify  the  program  request. 
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A 

2.  Automated  Usage. 

a.  The  basic  process  is  the  same  for  automated  as  for  manual  application; 
however,  much  greater  scope  and  depth  of  evaluation  can  be  accommodated  with 
computer  assistance.  To  this  end,  the  C BRA  computer  model  is  designed  to  be 
as  flexible  as  possible;  a  general-purpose  tool  which  can  be  used  in  whole  or 
in  part  for  many  different  types  of  tasks.  The  examples  presented  here  are 
representative  only,  ami  are  Intended  to  familiarize  the  user  with  the 
automated  technique  so  that  it  can  be  readily  adapted  to  "live"  needs  as  they 
are  encountered. 

b.  The  operating  instructions  in  this  Guide  are  keyed  to  the  specific 

IBM- Fortran  IV  computer  source  program  provided  at  Appendix  C.  However, 
utilization  is  by  no  means  limited  to  IBM/"main  frame"  type  computers.  In 
fact,  this  is  actually  a  relatively  cumbersorae/inflexible  program 
model — much  greater  flexibility /versatil l ty  can  be  obtained  from  smaller 
computers  designed  specifically  for  personal /interactive  use.  (One  such, 
ideally  suited,  is  the  Apple  II  with  Visl-Calc  package;  example  of  CBRA  -  Apple 
application  is  Included  with  IV. ^.2.)  The  sheer  diversity  and  versatility  of  I 

these  computers  renders  it  impractical  to  discuss  specific  instructions  in 

this  document;  but  the  manuals  furnished  with  each  manufacturer's  equipment 
make  the  "translation"  largely  self-explanatory  once  the  basic  technique  is 
understood . 

c .  Sample  Problems. 

(1)  Task.  Assess  the  relative  suitability  of  two  classes  of 
vehicles  for  performing  a  common  support  mission;  e.g.,  serving  as  a  target 
acquisition  &  designation  medium  for  offensive  weapon  systems.  Development  of 
a  new  system  specifically  designed  for  the  mission  is  precluded;  | 

therefore,  adaptation  of  existing  systems  mist  be  examined. 
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(2)  Critical  parameters.  Performance  of  the  target  acquisition/ 
designation  mission  entails  required  and  desired  capabilities  in  the  areas  of 
speed /power ,  endurance,  maneuverability,  detectability,  supportab  !1  i  ty , 
velnerabllity,  survivability/ self  defense,  comnunications  &  navigation,  crew 
workload,  and,  of  course,  the  vlslonics  package.  These  considerations  torra 
the  critical  parameters  to  be  addressed  in  the  evaluation. 

(3)  Evaluation  structure.  The  critical  parameters  fall  into  two 
basic  groups,  with  interrelationships  as  illustrated  following  page. 


(4)  Cell  weights.  The  weighting  process  should  address  the  areas  of 


greatest  contrast  among  candidates,  for  such  differences  are  the  reason  that 
tradeoff  analysis  is  needed.  In  this  case,  the  contrast  revolves  around 
vehicle  size  as  it  affects  required  capabilities;  i.e.,  the  larger-class 
vehicles  have  greater  power,  endurance  and  "payload"  capacity,  and  the 
smaller-class  vehicles  have  greater  maneuverability,  better  availability  (in 
the  specified  method  of  employment)  and  are  less  detectable.  This,  the  cell 
weights  must  reflect  the  relative  importance  of  these  aspects  to  isolate 
tradeoffs  and  areas  requiring  in-depth  examinat ion/ testing . 

Illustration  following  Page* 
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(5)  Scoring  standards.  Only  the  lowest-level  cells  (marked  with  *) 
are  assigned  raw  scores;  remaining  scores  are  computed  in  the  rollup  process. 


Following  standards  are  established 
supporting  rationale: 

CELL 

1.0  System 

1.1  Vehicle 

1.1.1  Performance 

*1.1. 1.1  Power 


SS 

1-3 

4-5 

6-7 

8-10 


(scale  of  1-10  points),  with 

RATIONALE 

Basic  vehicle  vs.  mission  equipment 
capability  tradeoff. 

Consider  performance,  detectability, 
crew  operation,  and  availability 
aspects. 

Key  aspects  are  power,  endurance  and 
maneuverab llity. 

Vehicle  must  have  sufficient  power  to 
operate  in  anticipated  battle  en¬ 
vironment,  enough  speed  to  be 
compatible  with  its  companion  (of¬ 
fensive  weapon)  systems  with  full 
mission  equipment  installed. 

Comes  within  2%  of  "required"  power 
ratings  and/or  speed  range. 

Meets  minlmums  requirements  with 
reasonable  confidence. 

Moderately  exceeds  some  requirements. 
Meets  all  requirements  and  sub¬ 
stantially  exceeds  (10%  or  better) 
some  aspects. 
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*1.1. 1.2  Maneuverability 


Mist  be  able  to  perform  on/over/in 


the  expected  battle  conditions 
(terrain,  altitudes,  temperatures, 
winds)  safely  and  easily  enough  to 
preclude  prolonged  exposure  to  th* 
enemy  while  keeping  pace  with  com¬ 
panion  v*»apon  systems. 

SS 

1-3  Performs  most  required  maneuvers,  but 

with  significant  payload/workload 
1 iml tat  lons/penal t  ies. 

4-5  Performs  maneuvers  but  with  little  or 

no  margin,  eltlier  for  vehicle  or  crew. 

6-7  Performs  responsively  and  with  reason¬ 

able  margin  In  one  or  more  aspects. 

8-10  Significantly  exceeds  minlmums  in 

several  aspects. 

*1.1. 1.3  Endurance  Must  be  able  to  perform  continuous 

mission  operations  without  scheduled 
out- of- service  tlnu,  for  3  hours; 

5  de  sired  . 


*1.1.2  Detectability 


SS 

1-3 


4-5 


6-7 


8-10 


Vehicle  must  be  able  to  "hide"  well 
enough  enroute  &  during  battle  so 
that  exposure  to  enemy  weapons  does 
not  permit  them  to  fix  location  and 
fire  while  in  kill  range.  Includes 
direct  visual,  radar,  and  IR  means  of 
detect  ton. 

Detectable  for  3-5  seconds  longer  than 
spec  Lfied  min.  guidelines  in  some 
modes/c ondi tions. 

Meets  established  minimums,  overall 
conditions. 

Meets  all  and  moderately  exceeds  some 
minim  ms. 

Not  detectable  long  enough  to  be 
successfully  acquired  at  any  point 
within  kill  range. 


*1.1.3  Crew  Workload 


3  Crew  Workload 

Overall  system  should  be  operable  by 

crew  of  two,  with  adequate  vehicle 

safety  margin  while  performing  mission 

maneuvers  and  employing  mission  equip¬ 
ment  . 

SS 

1-3 

Unsatisfactory  to  marginal. 

4-5 

Adequate  except  during  intense  battle 

&  adverse  visibility  conditions. 

6-7 

Potentially  adequate  with  extensive 

training  and/or  additional  skills. 

8-10 

Fully  adequate  with  normal  training 

&  skills. 

4  Availability 

Consider  impacts  of  logistic  support 

and  vulnerability  upon  overall 

readiness. 

4.1  Logistic  Support 

NORS/NOM/RAM  standards  equivalent  to 

System  X  are  desired;  at  minimum, 

characteristics  of  the  existing  Class 

A  &  B  Systems  should  not  be  degraded 

by  the  planned  modification/con¬ 
version. 

SS 

1-3 

Risk  of  minor  degradation. 

4-5 

Equivalent  to  status-quo  systems. 

6-7 

Improved  over  status  quo  but  less 

than  System  X. 

8-10 

Roughly  comparable  to  System  X. 
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*1.1. 4. 2  Vulnerability 


SS_ 

1-3 

4-5 

6-7 

8-10 

1.2  Mission  Equipment 

1.2.1.  Vision les 

*1.2.1. 1  Type 

SS 

1-3 

4-5 

6-7 


Consideration  of  expected  damage /loss 
during  normal  mission  operations, 
whether  resulting  from  enemy  fire, 
environmental  conditions,  handling 
problems,  or  other  causes. 

Significantly  more  vulnerable  than 
System  X. 

More  vulnerable  than  System  X  in 
some  aspects/conditions. 

Roughly  equivalent  to  System  X. 

Less  vulnerable  than  System  X. 

Consider  the  visionics,  comnunications , 
navigation,  and  survivability  equip¬ 
ment  needs. 

Addresses  type  of  acquisition  subsystem 
range  &  accuracy,  and  designation 
capab  ilities. 

Day/night/obscurants/all  season 
capability  is  desired. 

Day  only. 

Basic  day/nlght. 

Day /night  plus  certain  obscurants. 
Day/night/obscurant/al 1  weather 
subsystem. 


8-10 


1.2. 1,3  Range  &  Accuracy 


SS 

1-3 

4-5 

6-7 

8-10 

1.2. 1.3  Deslgnation/Handoff 


SS 

1-3 


4-5 


6-7 


To  achieve  safe  stand  off  ranges, 
following  detection/recognition 
assessment  pertains. 

Detect  (meters) _  Recognize 


1800  day/ 1200  night  75-80% 
2200  day/1800  night  80-85% 
3000  day/2200  night  85-90% 
4000  day/3000  night  90-95% 


Ability  to  designate  targets  for  sister 
weapon  systems,  throughout  the  above 
ranges,  plus  (desired)  capability  to 
automatically  hand  off  acquired 
targets  to  other  designation  systems. 


Designation  with  approx.  80%  success/ 
accuracy;  no  auto  handoff. 

Approx.  85%  designation  capability; 
still  no  auto  handoff. 

Approx.  90%  designation  success;  no 
auto  handoff. 


8-10 


Designation  90%  or  better,  auto 
handoff  capability. 


*1.2.2  Comma nlcatlons  & 
Navigation 

SS_ 

1-3 

4-5 

6-7 

8-10 

1.2.3  Survivability 


Full  suit  of  third- generation 
equipment  desired  (compatible  with 
System  X) . 

Can  accommodate  only  minimum  essential 
items  with  existing  spac e / we ight /power 
allowance. 

Can  accommodate  most  of  the  desired 
items,  but  entails  payload  tradeoffs/ 
human  factors  penalties. 

Can  accommodate  the  full  suit  with 
moderate  human  factors  Impacts. 

FU11  System  X  compatibility  with 
reasonable  crew  workload. 

Considers  ability  to  accommodate  IR 
suppression/countermeasures  equipment 
and  also  potential  for  self-defense 


armame  nt 


*1.2.3. 1  CM 


SS 

1-3 


4-5 

6-7 

8-10 

*1.2. 3. 2  Armament 


SS 

1-3 


4-5 


6-7 


Desired  features  include  radar 
warning,  1R  suppression,  IR  jamming, 
laser  warning  indicator,  NBC  device. 

Can  accommodate  only  radar  warning 
and  IR  suppression  items. 

Can  accommodate  above  plus  IR  jammer. 
Can  accommodate  all  except  NBC 
device,  but  with  some  performance/ 
payload  impacts. 

Can  accommodate  desired  items  with 
lLttle  to  no  impacts. 

Need  existing  space /weight /power 
allowance  for  first- generation  self 
defense  subsystem;  growth  potential 
for  second  generation  (multiple-type) 
self  defense  subsystem  is  desired. 

Existing  SWPA  (1st  gen)  with  mission 
equipment  tradeoffs  and  in  standard- 
day  condi tions . 

Existing  SWPA  (1st  gen)  with  full 
mission  equ ip. -min.  performance 
tradeoffs,  standard  day. 

Existing  SWPA  (1st  gen.)  to  include 
extreme  conditions. 


&-10 


Existing  SWPA  for  1st  gen  equipment 
plus  desired  growth  potential. 


(b)  Verify  candidates.  At  this  point,  the  "cund  id  a  te  s"  are  the  two 
classes  of  system  (A  &  B) ;  however,  based  oil  results  of  this  exercise,  similar 
structure  may  be  used  to  compare  specific  candidates  as  appropriate. 

(7)  Scor  ing.  Based  on  best  information  available,  assign  scores  based 
on  each  candidate's  expected  rating  for  each  critical  parameter  against  the 
established  scoring  standard.  (Docuiftjnt  sources  &  rationale.)  See  attached 
sheets  for  sample  System  A  &  B  scoring  input. 
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*1.2. 1.2  Range/Accuracy 


(8)  At  this  point  In  the  ease  exercise  (or  earlier,  if  desired)  it  is 
advisable  to  work  at  the  computer  terminal.  Therefore,  subsequent  instruc¬ 
tions  for  CBRA  computations  are  provided  in  the  DETAILED  OPERATING  INSTRUC¬ 
TIONS,  attached.  These  instructions  are  designed  to  lead  you  through  all 
basic  program  steps,  as  outlined  and  numbered  °n  CBRA  "Road  Map"  (flow  chart), 
included  with  Quick  Reference  Summary  at  IV. C. 


AUTOMATED  CBRA  -  DETAILED  INSTRUCTION 
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DO  YOU  WISH  TO  CHANGE  NUMBER  OF  CELLS  PER  LEVEL,  YES  OR  NO? 
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DO  YOU  WANT  TO  REPLACE  AN  OLD  DATA  SF.T  WITH  THIS  ONE,  YES  OR  NO? 
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Quick  Reference  Summary  is  next  (Section  IV. C),  followed  by  cost  and  cost-benefit  comparisons 


C.  QUICK  REFERENCE  SUMMARY 


1.  Basic  process  (manual  or  automated). 


a .  < 

CBRA  Technique-Ten  Steps* 

(i) 

Define  task. 

(2) 

Identify  critical  parameters  (CP) 

(3) 

Design  evaluated  structure  (ES) . 

(4) 

Allocate  cell  weights  (CW). 

(5) 

Establish  scoring  standards  (SS) . 

(6) 

Verify  evaluation  candidates. 

(7) 

Assign  raw  scores  (RS) ;  compute 
weighted  scores  (WS) . 

(8) 

Obtain  candidate  costs  (CC) . 

(9) 

Compute  cost-benefit  ratio  (CBR) . 

(10) 

Perform  analyses. 

b. 

Formula:  CW  x  RS  =  WS  +  CC  =  CBR. 

Automated 

CBRA  -  Flow  Chart  (next  paee"). 

* Document  rationale  throughout  the  process. 
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FLOW  PROCESS  CHART 


CBRA  TECHNIQUE  -  USER  S  GUIDE 
"ROAD  MAP" 


|  DIRECTIVE  NO 

OATE 

TR  AS1-F-: 

December  1980 

APPENDIX  NO 

SECTION  1V.C 


SUBJECT  OF  PROCESS 

Cost-Benefit  Ratio  Analysis  (CBRA)  Technique  -  Computer  Operation  Flow  Chart 


NARRATIVE  DESCRIPTIC*  Of  ACTION 


GRAPHIC  DESCRIPTION  Of  ACTIONS 


If  .ell  weights  add  properlv^at  all  levels, 

^er  skips  automatically  to  (26/  .  If  not,  go 

Data  ter  the  "problem  cell"  is  displayed  for 
to  determine  necessary  correction.  Go  to  (23)  . 

If  you  find  that  you  need  to  add  or  delete  a 
rather  than  revise  weights  of  existing  cells. 

;  "t*  and  go  back  to  QO)  .  Otherwise,  go  to  ^2M 

Enter  desired  CW  change  and  go  to  . 

If  changes  arc  complete,  hit  C/R;  computer  goes 
to  Ul)  and  continues  rollup.  If  more  changes 
needed  go  back  to  (2M  . 

When  edit/rollup  computations  for  all  levels  are 
lete,  enter  cell Ipse  for  ceil  1.0  (System 
-  lry)  and  go  to  ^7)  . 

If  you  wish  to  see  the  final  data  set,  tyre  YES 


if  you  wish 
so  to  @  ; 


if  not,  type  NC  and  skip  to 


The  final  data  is  displayed  for  ref erence.  but 
t  be  edited  further  in  thia  run.  Go  to  Q§)  . 

If  you  wish  to  atore  the  final  data  set,  either 
replacing- old  data  or  loading  new  file,  tvpe  YES 
to  ;  if  not,  type  NC  and  akip  to 

Enter  file  0  of  desired  storage  location;  go  tc 


/  \ 

FINAL  DAIAJ 

SET”*  jr 


DATA  SET 
D1SFLAYED 


STORE 


100? 

X'o 


r  PROBLEM 
DATA 

.DISPLAYED 


fES^  RETURN 
~  >wT0  EDIT’ 


li  duplicate  storage  is  desired,  type  YES  and  go 
tc  ;  if  not,  type  NO  and  go  to  ^3*)  . 

Advance  paper  tc  atart  freah  page  and  hit  RTN 
output.  Co  to  . 

Output  includes  annotation  cf  each  individual 
i*s  weight-contribution  to  the  total  system  weight 

If  you  wiah  to  make  another  run  (e,g.,  to  fill 
scores,  add  a  candidate,  perform  sensitivity 

sis,  etc.)  type  YES  and  go  back  to  (T^  .  If  not 
e  NO  and  go  to  .  v'— ^ 

Type  LOGOFF.  Computer  will  acknowledge.  Turn 
terminal . 


0,’m'T  \  & 

COMPUTATION/ 

DISPLAYED 


**Test  repeated  for 
each  level  until 
rollup  is  oonplete. 


YES 


MAKE 


PRECEDING  PAGE  BL*NK-NOT  FILMED 


D.  RESULTS:  ANALYSIS  AND  PRESENTATION 

1.  Review  of  the  output,  plus  consideration  of  candidate  costs,  may  lead  to 
several  angles  of  analysis — identification  of  acute  sensitivity/imbalance 
within  the  evaluation  structure  or  cell  weights,  anomalies  or  questions  in 
expected  data  relationships,  aspects  deserving  further  study — as  well  as 
preliminary  assessment  of  System  A  vs.  B  "bang  for  bucks".  There  are  probably 
as  many  ways  to  approach  the  review  and  presentation  of  data  as  there  are  in¬ 
dividuals;  so  in  this  case,  we'll  start  with  a  recap  of  the  data  itself: 

a.  Output  scores  for  Systems  A  &  B  under  both  versions  of  ES  weighting. 

Weighted  Weighted 


Raw  Scores 

CW-//1 

Scores 

CW-//2 

Scores 

A  B 

A  B 

A  B 

1  .0 

System  Summary 

100 

5S 

43 

100 

52 

43 

l.l 

Vehicle 

45 

39 

62 

55* 

35 

65 

1.1.1 

Pe  rformance 

35 

63 

37 

25* 

63 

37 

1.1. 1.1 

Power 

SO 

20 

30 

30 

1.1. 1.2 

Maneuverability 

45 

55 

40 

40 

1.1. 1.3 

Endu  ranee 

70 

30 

30 

30 

1.1.2 

Detectability 

20 

80 

30 

35* 

1.1.3 

Crew  Workload 

35 

65 

15 

20* 

1.1.4 

Availabil ity 

20 

26 

74 

20 

26 

74 

1. 1.4.1 

Logistic  Support 

40 

60 

45 

45 

1. 1.4.2 

Vii  lnerab  il  i  ty 

15 

85 

55 

55 

1.2 

Mission  Equipment 

55 

73 

27 

45* 

73 

27 

1.2.1 

Visionics  Package 

45 

74 

27 

45 

74 

27 

1.2. l.l 

Type  Subsystem 

SO 

20 

35 

35 

1.2. 1.2 

Range / Ac  cu racy 

70 

30 

40 

40 

1 . 2 . 1 . 3 

Designation/Handof  f 

70 

30 

25 

25 

1.2.2 

Comm/Nav iga  t ion 

70 

30 

30 

30 

1.2.3 

Survivability 

25 

75 

25 

75 

25 

1 .2.3.  t 

Cou  n  t  e  rme  a  su  r  e  s 

75 

25 

55 

55 

1.2. 3. 2 

Armament 

75 

25 

45 

45 

*Revised  weights 


c.  Observations: 


(1)  The  two  types  of  systems  being  explored  are  to  a  great  extent 
opposites  in  terms  of  attributes:  System  A  can  carry  more  capable  mission 
equipment  and  has  greater  speed  and  endurance,  but  is  weak  in  the  areas  of 
detectability,  maneuverability,  crew  workload,  and  vulnerability;  System  B 
tends  to  have  just  the  reverse  strengths  and  weaknesses.  This  is  an  expected 
relationship,  given  the  task  definition,  and  appears  to  be  reflected  in  the 
scoring  of  the  candidates. 

(2)  It  is  apparent  that  fairly  modest  changes  in  Vehicle  vs.  MEP 
cell  weights  have  a  fairly  significant  effect  on  outcome  -  particularly  so 
because  of  the  opposite-characteristic  nature  of  the  candidates.  In  this 
instance,  the  change  is  not  enough  to  reverse  the  overall  system  rank  order, 
but  enough  to  show  that  determination  of  Vehicle  vs.  MEP  importance  is 
critical  in  reaching  further  decisions. 

d.  Next,  a  look  at  estimated  candidate  costs  (constant  year  XX  dollars 
in  mil lions) . 


Sys  A 

Dev 

(Qty  500) 
Proc 

(Qty  501 
Acqn 

MEP 

69.0 

586.8 

655.9 

Vehicle 

68. A 

478.1 

546.5 

System 

137.4 

1064.9 

1202.3 

System  B 

MEP 

56.5 

547.3 

603.8 

Vehicle 

59.6 

355.6 

415.2 

System 

116.1 

902.9 

1019.0 

e.  Costs  are 

portrayed  graphically 

on  the  following  pages 

DEVELOPMENT  COST  COMPARISON 


Total  System 


Vehicle 


PROCUREMENT  COST  COMPARISON 
(Qty  500) 


Vehicle 


Total  System 


PRECEDINQ  Page  ELANK-NOT  FILMED 


7 

L 

A 


g.  Cost-benefit  ratio  computations. 

NOTE:  "Normalization"  of  data  applies  only  to  the  figures  being  directly 

compared;  i.e..  System  A  vs . B  for  each  cost  in  isolation.  If  other  normalized 
comparison  is  desired  (e.g..  Version  #1  vs.  Version  // 2),  it  may  be  computed 
from  the  data  on  this  sheet;  but  care  must  be  taken  to  avoid  "mixing  apples 
and  oranges." 


System  A  System  B 


Vehicle 

MEP 

System 

Vehicle 

MEP 

System 

Version  //I  Wts. 

Development:  Cost 

68 

69 

137 

60 

56 

116 

WS. 

39 

75 

58 

62 

25 

43 

Ratio 

1.74 

.92 

2.36 

96 

2.24 

270 

Normalized  CBR 

(1.81) 

(1.00) 

(1.00) 

(1.00) 

(2.43) 

(1.14) 

Procurement:  Cost 

478 

587 

1065 

356 

547 

903 

WS. 

39 

75 

58 

62 

25 

43 

Ratio 

12.26 

7.83 

18.36 

5.74 

21.88 

21.00 

Normalized  CBR 

(2.13) 

(1.00) 

(1.00) 

(1.00) 

(2.79) 

(1.14) 

Acquisition:  Cost 

346 

656 

1202 

415 

604 

1019 

WS. 

39 

75 

58 

62 

25 

43 

Ratio 

14.00 

8.75 

20.72 

6.69 

24.16 

23.10 

Normalized  CBR 

(209) 

(1.00) 

(1.00) 

(1.00) 

(2.76) 

(1.14) 

Version  //2  Wts. 

Development:  Cost 

68 

69 

137 

60 

56 

116 

WS. 

33 

75 

52 

65 

25 

48 

Ra  tio 

1.94 

.92 

2.63 

.92 

2.24 

2.42 

Normalized  CBR 

(2.10) 

(1.00) 

(1.09) 

(1.00) 

(2.43) 

(1.00) 

Procurement:  Cost 

473 

573 

1065 

356 

547 

903 

WS. 

35 

75 

52 

65 

25 

48 

Ratio 

13.7 

7.83 

20.48 

5.48 

21.88 

18.31 

Normalized  CBR 

(2.5) 

(1.00) 

(1.09) 

(1.00) 

(2.79) 

(1.00) 

Acquisition:  Cost 

546 

6  56 

1202 

415 

604 

1019 

WS. 

35 

75 

52 

65 

25 

48 

Ra  tio 

15.6 

3.75 

23.12 

6.38 

24.16 

21.23 

Normalized  CBR 

(2.45) 

(1.00) 

(1.09) 

(1.00) 

(2.76) 

(1.00 

h.  Graphic  portrayal  next  page 


COST  BENEFIT  RATIO  COMP ARISON  -  VERSION  01 


i.  Observations: 


(1)  The  impact  of  cell  weight  changes,  when  cost  is  included  in  the 
comparison,  is  significant  enough  to  alter  the  rank  order  of  the  candidates. 

(2)  However,  the  margins  (in  terms  of  total  system  outcome)  are  so 
narrow  for  both  benefit  and  C8R,  that  decisions  based  on  the  evaluation  so 
far  would  not  be  solidly  supportable. 

2.  So,  where  next?  Following  are  suggested  directions  for  further  analysis: 

a.  Deeper  examination  of  the  system  requirements  -  how  much  capability 
is  essential  to  mission  performance  for  this  system,  as  opposed  to  desirable? 
And  which  inspects  of  the  two  basic  categories  (vehicle  and  MEP)  should  be 
emphasizeu?  Perhaps  separate,  detail  sensitivity  analyses  in  these  two 
areas  would  shed  more  light  on  the  problem. 

b.  What  is  the  affordability  picture?  Is  it  possible  to  modify  either 
system  to  overcome  some  of  the  existing  limitations?  If  so,  how  would  this 
affect  cost?  (Be  sure  to  consider  the  possibility  of  spillover  technical 
ramifications,  too.) 

c.  Should  other  categories  of  cost  (e.g.,  operating  and  support  costs, 
force  costs)  be  considered? 

d.  In  the  areas  of  greatest  uncer tainity /sof test  data,  what  tests  or 
simulations  might  be  run  to  provide  better  information? 

3.  When  analysis  is  complete,  presentation  of  output/conclusions  can  be 
drawn  directly  from  the  data  and  charts  used  in  the  analysis.  Some  additional 
samples  are  included ,  following  pages. 


MILITARY  WORTH  VS  ACQUISITION  COST 


COST  $ 


TOTAL  WORTH  VS  TOTAL  (LCC)  COST 


COST  $*P 


IT  ACQUISITION  COST 


-OFFS 


A0-A142  521  COST  BENEFIT  RATIO  ANALYSIS  ( CBRA )  TECHNIQUE  A 

SIMPLIFIED  SYSTEM  EVALUAT I . . < U )  ARMY  AVIATION  RESEARCH 
AND  DEVELOPMENT  COMMAND  ST  LOUIS  MO  J  H  FITE  ET  AL . 
UNCLASSIFIED  DEC  80  USAAVRADCOM-TR-81 -F-2  F/G  14/1 


2/2- 


NL 


EVALUATION  PERSPECTIVES:  MISSION  SUITABILITY 


OPERATIONAL 


EVALUATION  PERSPECTIVES:  MILITARY  WORTH 


E.  TIPS  &  CAUTIONS. 


I .  •  Approach  &  Documentation. 

a .  Evaluation  criteria  and  scoring. 

(1)  Obviously,  it  is  desirable  to  utilize  the  most  objective 
possible  basis  for  measuring  and  scoring  in  the  evaluation  process.  Therefore, 
wherever  possible,  use  established  requirements/data  to  formulate  these 
aspects.  However,  whenever/wherever  hard  data  is  simply  not  available,  then 

go  to  the  technical  experts  and  obtain  opinions/ratings  to  use  in  these 
areas . 

(2)  Remember  that  the  purpose  of  the  CBRA  technique  is  to  yield 
more  systematic  basis  for  decisions/recommendations;  never  to  predispose  or 
limit  a  decision  itself.  in  this  regard,  a  "final  score"  has  no  more  meaning 
than  scores  for  isolated  segments  of  evaluation.  In  fact,  it  is  advantageous 
to  compute  cost  and/or  benefit  from  several  different  perspectives,  or  at 
several  different  levels,  to  yield  a  more  complete  picture  and  to  surface 
particularly  sensitive  facets  of  the  overall  evaluation. 

b.  Managing  the  problem.  In  a  large  or  complex  evaluation  structure, 
entailing  disparate  disciplines  or  categories  of  information,  it  may  be 
desirable  to  break  down  the  effort  into  "modules"  to  be  worked  on  initially 
by  separate  proponents.  It  is  then  relatively  easy  to  "marry  up"  these 
modules  when  you  are  ready,  so  long  as  each  has  an  identified  place  within 
the  overall  structure  (wiring  diagram). 

c.  Documentation .  Be  sure  to  define  and  document  the  assumptions, 
sources  of  information  and  rationale  applied.  This  is  always  important  in 
supporting  an  analysis,  but  is  particularly  so  in  using  an  informal,  quick- 
reaction  technique  such  as  CBRA. 
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(1)  Extra  care  should  be  taken  in  defining  qualitative  inputs 
and  the  criteria  for  dealing  with  them.  In  addition,  sensitivity  analyses, 
to  establish  "bands"  in  areas  of  uncertainty,  will  be  valuable  in  defusing 
challenges  based  on  nitpicky  disagreement  with  input/assumptions. 

(3)  Good  documentation  is  also  the  best  defense  against — real  or 
alleged — abuse  or  manipulation  of  results.  (Since  the  computation  process 
itself  is  easily  auditable,  clear  visibility  of  input  and  assumptions  leaves 
little  room  for  "mystery"  or  tempering  with  outcomes.) 

d.  A  philosophical  comment:  Whenever  possible  within  time  and  other 
constraints,  it  is  a  good  idea  to  involve  in  the  process  of  analysis  some  of  the 
disparate  elements  who  will  likely  review  or  react  to  the  outcome;  e.g.,  by 
participation  in  selection  of  critical  parameters,  evaluation  criteria, 
weighting,  or  even  scoring  of  intangible  aspects. 

(1)  This  approach  may  require  an  arbiter  or  referee*  but  pays 

off  in  terms  of  strengthening  of  the  evaluation  by  broadening  the  foundation  and 
by  fostering  better  appreciation  of  the  task  via  greater  exposure. 

(2)  It  also  reduces  the  opportunity  for  nonconstructive 
criticism,  since  the  major  players  will  have  become  at  least  "associate  team 
members"  by  virtue  of  having  contributed  to  the  game  plan. 


2 •  Software  Adapt at  ion . 


a.  The  CBRA  source  program  is  written  in  Fortran  IV  language  for  inter¬ 
active  use  on  an  IBM  360/65  computer,  accessible  from  a  remote  terminal.  It 
is  easily  transportable  to  any  computer  which  utilizes  a  Fortran  compiler. 

b.  If  you  have  access  to  personal-interactive  type  computers,  better 
results  (and  virtually  infinite  variations)  can  be  attained  by  merely  applying 
the  CBRA  flow  chart  logic  IAW  the  specific  computer  operating  instructions, 
rather  than  attempting  to  actually  convert  the  program.  (Some  sample  output, 
using  Apple  II  with  Visi-Calc  package,  is  attached.)  It  should  also  be  noted 
that  this  type  of  computer  application  is  much  faster  than  "main  frame"  type 
applicat ion . 

c.  If  you  encounter  difficulty  in  adapting  the  CBRA  model  to  your  situation, 
and  cannot  obtain  needed  programming  assistance  locally,  use  the  Questionnaire 
provided  at  V.D  to  contact  the  authors  for  assistance. 
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APPLE 


II  UERSION  uF  CBRH  MODEL 
ADAPTED  SEPTEMBER  I960  BV 


USING  _ UISICALC  AS 
MAJ  JAMES  p.  bell 


SOFTWARE  BASIS 
ASH  PMO 


/  FuF.mhI  1 


PR  1  fit  LEUEL  1  SCORE  u 

SOB 


loot  A 

L  ELL  4 

CELL  ILi 

1  •hLUE 

WEIGHT 

INDEX 

CELL  #  CELL  ID 

UALUE 

WEIGHT 

1 

,.11  OO0O 

TtLH 

0 

40 

•’3 

XI 30000  LUb  SUPT 

0 

15 

.* 

1  1  1 000 

H'l  PERK 

0 

35 

74 

131000  TRANSPi'iF: 

0 

15 

3 

1  1  1  1  Ok) 

UROC 

0 

15 

75 

131100  C-5 

0 

15 

4 

1 1 1 200 

AIR  bPO 

0 

15 

76 

131200  C-130 

0 

25 

5 

1 1 1 300 

ENUURANC 

0 

20 

77 

131300  C- 141 

0 

50 

b 

1 1 1 400 

HGlL 'MAN 

0 

50 

78 

131400  SELF 

o 

10 

7 

1 1 1410 

LOW  bPEO 

0 

35 

79 

132000  SUPV  bPT 

0 

25 

y 

111420 

cunt  our 

0 

40 

80 

132100  INT  PROA 

0 

50 

9 

1 1 1430 

LANDING 

0 

25 

81 

132200  REPLNSH 

0 

35 

10 

112000 

JMl 

0 

35 

82 

132300  COMMUNAL 

0 

15 

1 1 

112100 

TGT  DET 

0 

25 

83 

0 

25 

12 

1 121 10 

DAY 

0 

50 

84 

133100  MTBF 

0 

50 

13 

112120 

NIGHT 

0 

40 

85 

133110  MEP 

0 

60 

14 

1 12130 

OBSCURNT 

U 

10 

88 

133120  AIRCRAFT 

0 

40 

15 

1 12200 

TGT  RECO 

0 

25 

87 

133200  MMH/'FH 

0 

50 

It 

1 12210 

DAY 

0 

50 

88 

133210  AUUM 

0 

40 

17 

1 12220 

NlbHT 

0 

40 

89 

133211  MEPXXXXX 

0 

50 

IB 

1 12230 

O&kC'JRNT 

0 

10 

90 

133212  AIRCRAFT 

0 

50 

IS 

1 12300 

Tur  uEgI 

0 

20 

SI 

133220  AUIM 

0 

60 

20 

1 12310 

DAY 

0 

35 

92 

133221  MEP 

0 

60 

'l 

1 1 2  320 

NIGHT 

0 

30 

b3 

133222  AIRCRAFT 

0 

70 

y  -• 

1 12330 

INTEROP 

o 

35 

94 

U400LI  HLl  IDENT 

0 

c 

23 

1 12400 

Tb  r  LUC 

0 

20 

95 

135000  MAT  IHTR  0 

30 

24 

1 12410 

ACi  IJRAC-V 

0 

60 

96 

135100  PERSONAL 

0 

30 

25 

1 12420 

FEATURES 

0 

40 

8  7 

135200  SKILLS 

O 

15 

2b 

1 12500 

P  FEEBBK 

0 

10 

98 

135300  TRAINlNb 

0 

15 

27 

1 12510 

DISPLAY 

0 

60 

89 

135400  DATA/'MAN 

0 

30 

2b 

1 12520 

RECORD 

0 

40 

100 

135500  TOE  CHGS 

0 

10 

2S 

l 13000 

AT HR  MEP 

0 

10 

101 

X 140000  ATTAINAB  0 

10 

30 

113100 

COMHO  EO 

0 

40 

102 

141000  ACQN  APP 

0 

35 

31 

1 1 3200 

NAD  EG 

0 

50 

103 

141100  DEDELuPM 

0 

30 

32 

1 13300 

ASE  EO 

0 

10 

104 

141200  PRODUCT I 

0 

40 

33 

JRHHttfcNI 

0 

5 

105 

141300  OUERHAUL 

0 

10 

34 

115000 

SURA I DAB 

0 

C. 

.Jt 

108 

141400  $  TIMING 

0 

20 

35 

1 15100 

CRASHAOR 

0 

30 

107 

142000  RISK 

0 

55 

3b 

1 15200 

UULNEKAfc 

0 

70 

108 

142100  SCHEDULE 

0 

35 

37 

115210 

PH/PK 

0 

20 

109 

142110  IOC 

0 

65 

38 

1 15220 

STRIKE 

Ci 

40 

1 10 

142120  CONCURRE 

0 

35 

39 

115230 

BAl.L  1ST  I 

0 

40 

111 

142200  TECHNICA 

0 

40 

40 

115231 

CREN 

0 

60 

112 

142210  MEP 

0 

25 

41 

1 15232 

fl  r  cunt 

0 

40 

113 

142220  SYS  IMTE 

0 

35 

42 

1  IftftHH 

jfiftUHTH  P 

0 

10 

114 

142230  STE/PbSE 

0 

10 

43 

1 16100 

ARMAMENT 

0 

35 

115 

142240  PRODUCIB 

0 

15 

44 

1 16100 

MEP 

0 

55 

1  lb 

142250  fi'C  PERF 

0 

15 

45 

1  lb  3oo 

SUM'  >  I  UhB 

0 

10 

117 

142300  COST  REA 

0 

25 

4b 

1 16400 

A /i  PERF 

0 

20 

1 18 

142310  DEUELOPM 

0 

40 

47 

1 16410 

ARUb 

0 

20 

1 19 

142320  PROCUREH 

0 

60 

48 

1 16420 

Da  -h 

0 

10 

120 

143000  MGT  PLAN  9 

10 

4S 
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ASF:  Army  Stock  Fluid 

APA:  Aircraft  Procurement  Army 

BCE:  Baseline  Cost  Estimate 

CA:  Cost  Analysis 

CB:  Cell  Block 

CBRA:  Cost  Benefit  Ration  Analysis 

CBR:  Cost  Benefit  Ratio 

CC:  Candidate  Cost 

CCM:  Counter-Countermeasures 

CFP:  Concept  Formulation  Package 

CM:  Countermeasures 

COEA:  Cost  &  Operational  Effectiveness  Analysis 

CP:  Critical  Parameter 

CTEA:  C  js  t  &  Training  Effectiveness  Analysis 

CW:  Cell  Weight 

EA:  Economic  Analysis 

ECP:  Engineering  Change  Proposal 

ED:  Engineering  Devolopmet 

EPA:  Extended  Planning  Annex 
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ES: 


Evaluation  Structure 


EW:  Electronic  Warfare 

FSED:  Full  Scale  Engineering  Development 

LCC:  Life  Cycle  Cost 

LCCE:  Life  Cycle  Cost  Estimate 

MTBF:  Mean  Time  Between  Failure 

OMA:  Operations  &  Maintenance  Army 

O&S  Operating  &  Support 

PIF:  Product  Improvement  Program 

POM:  Program  Objective  Memorandum 

RA:  Risk  Assessment 

RAM:  Reliability  and  Maintainability 

RDTE:  Research,  Development,  Test  &  Evaluation 

RISNET:  Risk  Information  System  &  Network  Evaluation  Technique 

RS:  Raw  Score 

RW:  Relative  Worth 

SA:  Sensitivity  Analysis 

SS:  Scoring  Standard 

S&E:  Scientific  &  Engineering 

SSEB:  Source  Selection  Evaluation  Board 

TOA  ($) :  Total  Obligation  Authority 

TOA:  Tradeoff  Analysis 

TOD:  Tradeoff  Determination 

TRACE:  Total  Risk  Assessing  Cost  Estimate 
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Weighted  Score 
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VI 


DISTRIBUTION 


Qty 


Headquarters,  Department  of  the  Army 

ATTN:  DAMA-WS  (Mr.  T.  Miller)  1 

DAMA-WSA  (Mr.  R.  Ballard)  2 

DAMA-PP  1 

DACA-CA  (Mr.  Luker)  1 

Washington,  DC  20310 

Commander 

US  Army  Materiel  Development  and  Readiness  Command 
ATTN:  DRCPI  (Ms.  Hughes)  1 

DRCCP-EM  1 

DRCMM-RM  (Mr.  Phillips)  1 

DRCDE-DH  (COL  Ramey)  1 

DRCPM  6 

3001  Eisenhower  Avenue 
Alexandria,  VA  22333 


Commandant 

Defense  Systems  Management  College 

ATTN:  DSMC-SEL-L  (LTC  Bohls)  5 

Fort  Belvoir ,  VA  22060 

Commander 

Automated  Logistics  Management  Systems  Agency 
ATTN:  DRXAL-FDB  (Mayne)  1 

210  North  12th  Street 
St.  Louis,  MO  63101 

Director 

US  Army  Research  &  Technology  Laboratories 
Ames  Research  Center 

ATTN:  Mike  Scully  1 

Mail  Stop  207-5 

Moffett  Field,  CA  94035 

Commander 

US  Army  Training  and  Doctrine  Command 

ATTN:  ATCD-AN-R  12 

ATCD-SM 

Fort  Monroe,  VA  23651 

with  suggested  dissemination  to: 

ATZK-XM1 

ATSA-TSM-G 

ATZH-TSM-ATC 

ATSH-TSM-FV 

ATZQ-D 

ATZQ-TSM-S 

ATZL-CAM-IC 

Others  as  appropriate 
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Distribution  Continued 


Commander 

US  Army  Troop  Support  &  Aviation  Materiel  Readiness  Command 
ATTN:  DRSTS-W 
DRSTS-C 

4300  Goodfellow  Boulevard 
St.  Louis,  MO  63120 

Commander 

US  Army  Aviation  Research  &  Development  Command 
ATTN:  DRDAV-N 
DRDAV-B 
DRDAV-EX 

4300  Goodfellow  Boulevard 
St.  Louis,  MO  63120 

Project  Manager 
Cobra 

ATTN:  DRCPM-CO 

4300  Goodfellow  Boulevard 

St.  Louis,  MO  63120 

Project  Manager 

Special  Electronic  Mission  Aircraft 
ATTN:  DRCPM-AE 
4300  Goodfellow  Boulevard 
St.  Louis,  MO  63120 

Project  Manager 

Aircraft  Survivability  Equipment 
ATTN:  DRCPM-ASE 
4300  Goodfellow  Boulevard 
St.  Louis,  MO  63120 

Project  Manager 

CH-47  Modernization  Program 

ATTN:  DRCPM-CH47M 

4300  Goodfellow  Boulevard 

St.  Louis,  MO  63120 

Project  Manager 

Black  Hawk 

ATTN:  DRCPM-BH 

4300  Goodfellow  Boulevard 

St.  Louis,  MO  63120 
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fRECEDINQ  PAGE  Ri  .AMK-NOT  FILMED 


A 


VII  -  CBRA  TECHNIQUE  FEEDBACK  QUESTIONNAIRE 


From: 


Send  to:  Cdr,  USAAVRADCOM 

ATTN:  DRCPM-ASH-S 
4300  Goodfellow  Blvd. 

St.  Louis,  MO  63120 

Or  call:  Autovon  693-1368,  Mrs.  Fite,  for  overall  questions/comments . 

Autovon  693-1179,  Mr.  Oxandale,  for  computer  application  aspects. 

1.  CBRA  Technique  User's  Guide  Z~~7  has,  /~7  has  not,  been  reviewed. 


2.  _  copy(ies)  hereby  returned,  as  not  needed/desired  by  this  organization. 

3.  _  additional  copy(ies)  requested  for  further  dissemination/utilization. 

4.  Following  additional  information/clarification  is  requested.  (Continue  on 
additional  pages  if  needed.) 


5.  This  organization  ZZ7  has  applied/adapted, O  is  considering  utilization/adapta¬ 
tion  of  CBRA  Technique,  in  the  following  instances/areas : 


6.  Based  on  ZZ7  review,  £J  utilization  of  the  Guide,  following  comments/suggest ions 
for  improvement  are  offered.  (Continue  on  additional  pages  if  needed.) 
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